Application of artificial intelligence to eyewitness identification.

Artificial intelligence is already all around us, and its usage will only increase. Knowing its capabilities is critical. A facial recognition system (FRS) is a tool for law enforcement during suspect searches and when presenting photos to eyewitnesses for identification. However, there are no comparisons between eyewitness and FRS accuracy using video, so it is unknown whether FRS face matches are more accurate than eyewitness memory when identifying a perpetrator. Ours is the first application of artificial intelligence to an eyewitness experience, using a comparative psychology approach. As a first step to test system accuracy relative to eyewitness accuracy, participants and an open-source FRS (FaceNet) attempted perpetrator identification/match from lineup photos (target-present, target-absent) after exposure to real crime videos with varied clarity and perpetrator race. FRS used video probe images of each perpetrator to achieve similarity ratings for each corresponding lineup member. Using receiver operating characteristic analysis to measure discriminability, FRS performance was superior to eyewitness performance, regardless of video clarity or perpetrator race. Video clarity impacted participant performance, with the unclear videos yielding lower performance than the clear videos. Using confidence-accuracy characteristic analysis to measure reliability (i.e., the likelihood the identified suspect is the actual perpetrator), when the FRS identified faces with the highest similarity values, they were accurate. The results suggest FaceNet, or similarly performing systems, may supplement eyewitness memory for suspect searches and subsequent lineup construction and knowing the system's strengths and weaknesses is critical.

pyWitness 1.0: A python eyewitness identification analysis toolkit.

pyWitness is a python toolkit for recognition memory experiments, with a focus on eyewitness identification (ID) data analysis and model fitting. The current practice is for researchers to use different statistical packages to analyze a single dataset. pyWitness streamlines the process. In addition to conducting key data analyses (e.g., receiver operating characteristic analysis, confidence accuracy characteristic analysis), statistical comparisons, signal-detection-based model fits, simulated data generation, and power analyses are also possible. We describe the package implementation and provide detailed instructions and tutorials with datasets so that users can follow. There is also an online manual that is regularly updated. We developed pyWitness to be user-friendly, reduce human interaction with pre-processing and processing of data and model fits, and produce publication-ready plots. All pyWitness features align with open science practices, such that the algorithms, fits, and methods are reproducible and documented. While pyWitness is a python toolkit, it can also be used from R for users more accustomed to this environment.

Illustrations of interactions needed when investigating sleep using a type of AM-PM PM-AM design.

Sleep has long been thought of and promoted to be beneficial for memory. Some claims that sleep aids memory have been made in the absence of a critical interaction. This condition is necessary when using a commonly-used experimental design (a type of AM-PM PM-AM design). We propose that a sleep effect exists only if there is an interaction between groups (experimental and time-of-day controls) and the time of test or study (morning and evening). We show different patterns of results that would and would not support a sleep effect with empirical and model-generated data from recognition memory experiments and hypothetical data. While we use these data to make our point, our suggestions apply to any memory and non-memory-related investigation (e.g., emotional memory, false memory susceptibility, language learning, problem-solving). Testing for and finding the proper interaction will add to the evidence that sleep boosts performance.

Lineup identification in young and older witnesses: does describing the criminal help or hinder?

The world population is getting older and, as a result, the number of older victims of crime is expected to increase. It is therefore essential to understand how ageing affects eyewitness identification, so procedures can be developed that enable victims of crime of all ages to provide evidence as accurately and reliably as possible. In criminal investigations, witnesses often provide a description of the perpetrator of the crime before later making an identification. While describing the perpetrator prior to making a lineup identification can have a detrimental effect on identification in younger adults, referred to as verbal overshadowing, it is unclear whether older adults are affected in the same way. Our study compared lineup identification of a group of young adults and a group of older adults using the procedure that has consistently revealed verbal overshadowing in young adults. Participants watched a video of a mock crime. Following a 20-min filled delay, they either described the perpetrator or completed a control task. Immediately afterwards, they identified the perpetrator from a lineup, or indicated that the perpetrator was not present, and rated their confidence. We found that describing the perpetrator decreased subsequent correct identification of the perpetrator in both young and older adults. This effect of verbal overshadowing was not explained by a change in discrimination but was consistent with participants adopting a more conservative criterion. Confidence and response time were both found to predict identification accuracy for young and older groups, particularly in the control condition.

Eyewitness memory is reliable, but the criminal justice system is not.

The reliability of any type of forensic evidence (e.g., forensic DNA) is assessed by testing its information value when it is not contaminated and is properly tested. Assessing the reliability of forensic memory evidence should be no exception to that rule. Unfortunately, testing a witness's memory irretrievably contaminates it. Thus, only the first (properly conducted) test is relevant to the question of whether eyewitness memory is reliable. With few exceptions, the results of studies conducted in the lab and in the real world show that confidence is highly predictive of accuracy on the first test, and high-confidence often implies high accuracy. The fact that many eyewitnesses are known to have made high-confidence misidentifications in the courtroom has cemented the almost universal impression that eyewitness memory is unreliable. However, it is the criminal justice system that is guilty of unwittingly using contaminated memory evidence (relying on the last memory test, in court) in conjunction with an improper testing procedure (namely, a courtroom showup) to win convictions of the innocent. That mistake should no longer be blamed on the unreliability of eyewitness memory.

Doing right by the eyewitness evidence: a response to Berkowitz et al.

Berkowitz et al. (Berkowitz, S. R., Garrett, B. L., Fenn, K. M., & Loftus, E. F. (2020). Convicting with confidence? Why we should not over-rely on eyewitness confidence. Memory. https://doi.org/10.1080/09658211.2020.1849308) attribute to us the claim that "confidence trumps all", and the few out-of-context quotations they selected can certainly be used to create that false impression. However, it is easily disproved, and we do so here. The notion that "confidence trumps all" is the mistake that the jurors made in the DNA exoneration cases, not a position that we have ever advocated.

Eyewitness Identification Is a Visual Search Task.

The simultaneous six-pack photo lineup is a standard eyewitness identification procedure, consisting of one police suspect plus five physically similar fillers. The photo lineup is either a target-present array (the suspect is guilty) or a target-absent array (the suspect is innocent). The eyewitness is asked to search the six photos in the array with respect to a target template stored in memory (namely, the memory of the perpetrator's face). If the witness determines that the perpetrator is in fact in the lineup (detection), then the next step is to specify the position of the perpetrator's face in the lineup (localization). The witness may also determine that the perpetrator is not present and reject the lineup. In other words, a police lineup is a detection-plus-localization visual search task. Signal detection concepts that have long guided thinking about visual search have recently had a significant impact on our understanding of police lineups.

The effect of lineup size on eyewitness identification.

Eyewitness identification via lineup procedures is an important and widely used source of evidence in criminal cases. However, the scientific literature provides inconsistent guidance on a very basic feature of lineup procedure: lineup size. In two experiments, we examined whether the number of fillers affects diagnostic accuracy in a lineup, as assessed with receiver-operating characteristic (ROC) analysis. Showups (identification procedures with one face) led to lower discriminability than simultaneous lineups. However, in neither experiment did the number of fillers in a lineup affect discriminability. We also evaluated competing models of decision-making from lineups. This analysis indicated that the standard Independent Observations (IO) model, which assumes a decision rule based on the comparison of memory strength signals generated by each face in a lineup, is incapable of reproducing the lower level of performance evident in showups. We could not adjudicate between the Ensemble model, which assumes a decision rule based on the comparison of the strength of each face with the mean strength across the lineup, and a newly introduced Dependent Observations model, which adopts the same decision rule as the IO model, but with correlated signals across faces. We draw lessons for users of lineup procedures and for basic research on eyewitness decision making. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Designing police lineups to maximize memory performance.

How can lineups be designed to elicit the best achievable memory performance? One step toward that goal is to compare lineup procedures. In a recent comparison of U.S. and U.K. lineup procedures, discriminability and reliability was better when memory was tested using the U.S. procedure. However, because there are so many differences between the procedures, it is unclear what explains this superior performance. The main goal of the current research is, therefore, to systematically isolate the differences between the U.S. and U.K. lineups to determine their effects on discriminability and reliability. In 5 experiments, we compared (a) presentation format: simultaneous versus sequential; (b) stimulus format: photos versus videos; (c) number of views: 1-lap versus 2-lap versus choice in both video and photo lineups; and (d) lineup size: 6- versus 9-lineup members. Most of the comparisons did not show appreciable differences, but one comparison did: simultaneous presentation yielded better discriminability than sequential presentation. If the results replicate, then policymakers should recommend using a simultaneous lineup procedure. Moreover, consistent with previous research, identifications made with high confidence were higher in reliability than identifications made with low confidence. Thus, official lineup protocols should require collecting confidence because of the diagnostic value added. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Models of lineup memory.

Face recognition memory is often tested by the police using a photo lineup, which consists of one suspect, who is either innocent or guilty, and five or more physically similar fillers, all of whom are known to be innocent. For many years, lineups were investigated in lab studies without guidance from standard models of recognition memory. More recently, signal detection theory has been used to conceptualize lineup memory and to motivate receiver operating characteristic (ROC) analysis of lineup performance. Here, we describe three competing signal-detection models of lineup memory, derive their likelihood functions, and fit them to empirical ROC data. We also introduce the notion that memory signals generated by the faces in a lineup are likely to be correlated because, by design, those faces share features. The models we investigate differ in their predictions about the effect that correlated memory signals should have on the ability to discriminate innocent from guilty suspects. A popular compound signal detection model known as the Integration model predicts that correlated memory signals should impair discriminability. Empirically, this model performed so poorly that, going forward, it should probably be abandoned. The best-fitting model incorporates a principle known as "ensemble coding," which predicts that correlated memory signals should enhance discriminability. The ensemble model aligns with a previously proposed theory of eyewitness identification according to which the simultaneous presentation of faces in a lineup enhances discriminability compared to when faces are presented in isolation because it permits eyewitnesses to detect and discount non-diagnostic facial features.

In the DNA Exoneration Cases, Eyewitness Memory Was Not the Problem: A Reply to Berkowitz and Frenda (2018) and Wade, Nash, and Lindsay (2018).

The available real-world evidence suggests that, on an initial test, eyewitness memory is often reliable. Ironically, even the DNA exoneration cases-which generally involved nonpristine testing conditions and which are usually construed as an indictment of eyewitness memory-show how reliable an initial test of eyewitness memory can be in the real world. We endorse the use of pristine testing procedures, but their absence does not automatically imply that eyewitness memory is unreliable.

Rethinking the Reliability of Eyewitness Memory.

Although certain pockets within the broad field of academic psychology have come to appreciate that eyewitness memory is more reliable than was once believed, the prevailing view, by far, is that eyewitness memory is unreliable-a blanket assessment that increasingly pervades the legal system. On the surface, this verdict seems unavoidable: Research convincingly shows that memory is malleable, and eyewitness misidentifications are known to have played a role in most of the DNA exonerations of the innocent. However, we argue here that, like DNA evidence and other kinds of scientifically validated forensic evidence, eyewitness memory is reliable if it is not contaminated and if proper testing procedures are used. This conclusion applies to eyewitness memory broadly conceived, whether the test involves recognition (from a police lineup) or recall (during a police interview). From this perspective, eyewitness memory has been wrongfully convicted of mistakes that are better construed as having been committed by other actors in the legal system, not by the eyewitnesses themselves. Eyewitnesses typically provide reliable evidence on an initial, uncontaminated memory test, and this is true even for most of the wrongful convictions that were later reversed by DNA evidence.

Theoretical vs. empirical discriminability: the application of ROC methods to eyewitness identification.

Receiver operating characteristic (ROC) analysis was introduced to the field of eyewitness identification 5 years ago. Since that time, it has been both influential and controversial, and the debate has raised an issue about measuring discriminability that is rarely considered. The issue concerns the distinction between empirical discriminability (measured by area under the ROC curve) vs. underlying/theoretical discriminability (measured by d' or variants of it). Under most circumstances, the two measures will agree about a difference between two conditions in terms of discriminability. However, it is possible for them to disagree, and that fact can lead to confusion about which condition actually yields higher discriminability. For example, if the two conditions have implications for real-world practice (e.g., a comparison of competing lineup formats), should a policymaker rely on the area-under-the-curve measure or the theory-based measure? Here, we illustrate the fact that a given empirical ROC yields as many underlying discriminability measures as there are theories that one is willing to take seriously. No matter which theory is correct, for practical purposes, the singular area-under-the-curve measure best identifies the diagnostically superior procedure. For that reason, area under the ROC curve informs policy in a way that underlying theoretical discriminability never can. At the same time, theoretical measures of discriminability are equally important, but for a different reason. Without an adequate theoretical understanding of the relevant task, the field will be in no position to enhance empirical discriminability.

The role of estimator variables in eyewitness identification.

Estimator variables are factors that can affect the accuracy of eyewitness identifications but that are outside of the control of the criminal justice system. Examples include (1) the duration of exposure to the perpetrator, (2) the passage of time between the crime and the identification (retention interval), (3) the distance between the witness and the perpetrator at the time of the crime. Suboptimal estimator variables (e.g., long distance) have long been thought to reduce the reliability of eyewitness identifications (IDs), but recent evidence suggests that this is not true of IDs made with high confidence and may or may not be true of IDs made with lower confidence. The evidence suggests that though suboptimal estimator variables decrease discriminability (i.e., the ability to distinguish innocent from guilty suspects), they do not decrease the reliability of IDs made with high confidence. Such findings are inconsistent with the longstanding "optimality hypothesis" and therefore require a new theoretical framework. Here, we propose that a signal-detection-based likelihood ratio account-which has long been a mainstay of basic theories of recognition memory-naturally accounts for these findings. (PsycINFO Database Record

The effects of verbal descriptions on performance in lineups and showups.

Verbally describing a face has been found to impair subsequent recognition of that face from a photo lineup, a phenomenon known as the verbal overshadowing effect (Schooler & Engstler-Schooler, 1990). Recently, a large direct replication study successfully reproduced that original finding (Alogna et al., 2014). However, in both the original study and the replication studies, memory was tested using only target-present lineups (i.e., lineups containing the previously seen target face), making it possible to compute the correct identification rate (correct ID rate; i.e., the hit rate) but not the false identification rate (false ID rate; i.e., the false alarm rate). Thus, the lower correct ID rate for the verbal condition could reflect either reduced discriminability or a conservative criterion shift relative to the control condition. In four verbal overshadowing experiments reported here, we measured both correct ID rates and false ID rates using photo lineups (Experiments 1 and 2) or single-photo showups (Experiments 3 and 4). The experimental manipulation (verbally describing the face or not) occurred either immediately after encoding (Experiments 1 and 3) or 20-min after encoding (Experiments 2 and 4). In the immediate condition, discriminability did not differ between groups, but in the delayed condition, discriminability was lower in the verbal description group (i.e., a verbal overshadowing effect was observed). A fifth experiment found that the effect of the immediate-versus-delayed manipulation may be attributable to a change in the content of verbal descriptions, with the ratio of diagnostic to generic facial features in the descriptions decreasing as delay increases. (PsycINFO Database Record

US line-ups outperform UK line-ups.

In the USA and the UK, many thousands of police suspects are identified by eyewitnesses every year. Unfortunately, many of those suspects are innocent, which becomes evident when they are exonerated by DNA testing, often after having been imprisoned for years. It is, therefore, imperative to use identification procedures that best enable eyewitnesses to discriminate innocent from guilty suspects. Although police investigators in both countries often administer line-up procedures, the details of how line-ups are presented are quite different and an important direct comparison has yet to be conducted. We investigated whether these two line-up procedures differ in terms of (i) discriminability (using receiver operating characteristic analysis) and (ii) reliability (using confidence-accuracy characteristic analysis). A total of 2249 participants watched a video of a crime and were later tested using either a six-person simultaneous photo line-up procedure (USA) or a nine-person sequential video line-up procedure (UK). US line-up procedure yielded significantly higher discriminability and significantly higher reliability. The results do not pinpoint the reason for the observed difference between the two procedures, but they do suggest that there is much room for improvement with the UK line-up.

The effect of instructor fluency on students' perceptions of instructors, confidence in learning, and actual learning.

Students' judgments of their own learning are often misled by perceptions of fluency-the ease with which information is presented during learning. Lectures represent important learning experiences that contain variations in fluency, but have not been extensively studied. In the current study, students watched a 22-min videotaped lecture that was delivered by the same instructor in either a fluent (strong, confident, and deliberate) manner, or in a disfluent (uncertain, hesitant, and disengaged) manner. Students then predicted their score on an upcoming test on the information, rated the instructor on traditional evaluation measures, and took a multiple-choice test on the information immediately (Experiment 1), after 10 min (Experiment 2), or after 1 day (Experiment 3). The fluent instructor was rated significantly higher than the disfluent instructor, but test scores did not consistently differ between the 2 conditions. Though students did not indicate higher confidence overall in learning from a fluent instructor, Experiment 3 found that when participants base their confidence on the instructor, those in the fluent condition were more likely to be overconfident. These findings indicate that instructor fluency leads to higher ratings of instructors and can lead to higher confidence, but it does not necessarily lead to better learning. (PsycINFO Database Record